feat(transcription): GPU execution provider with CPU fallback (Track-B step 6)#14
Open
farce1 wants to merge 1 commit into
Open
feat(transcription): GPU execution provider with CPU fallback (Track-B step 6)#14farce1 wants to merge 1 commit into
farce1 wants to merge 1 commit into
Conversation
Pick the ONNX Runtime execution provider per platform (CoreML on macOS, CPU elsewhere) instead of hardcoding CPU, and retry on CPU if a non-CPU provider fails to load so transcription never regresses. Both the selection and the fallback policy are unit-tested.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Track-B step 6 — GPU execution provider with CPU fallback
WhisperEngine/ParakeetEnginepreviously hardcoded the ONNX Runtime providerto
"cpu". This selects a platform-appropriate execution provider instead —CoreML on macOS, CPU elsewhere — and retries on CPU if a non-CPU provider
fails to load, so transcription can never regress relative to the CPU baseline.
Stacked on
feat/asr-model-picker(#13):#9 ← #10 ← #11 ← #12 ← #13 ← this.Design notes
sherpa-rsis built with onlydownload-binaries(nocudafeature), so the prebuilt binaries are CPU-only off macOS; CoreML is the only GPU EP this build can use. CUDA selection would be dead code, so it's intentionally omitted (YAGNI) until a CUDA-enabled build is introduced.select_provider(os)(pure) andwith_cpu_fallback(provider, build)(the safety policy) are both unit-tested;load_with_providercarries the real recognizer config.Gate
BE: clippy clean,
cargo build✓,cargo test --lib12 passed (the only 3 failures are the inherited maintainer-ownedREPLACE_WITH_SHA256 release gate). FE: unchanged on this branch.Merges once
main's vendor +REPLACE_WITH_blockers are cleared and the chain rebases.